cl: support Gloas alpha.12 progressive SSZ and spectests - #22912
Conversation
ee53979 to
2ca0558
Compare
|
Reviewed the full diff. Approve with nits. The hashing core is sound and I could verify it independently: the light-client branch sizes reproduce from Should fix
panic(fmt.Sprintf("Can't create TreeRoot: unsported type %T at index %d", i, obj))
Worth discussingThe new spectest handlers reimplement the logic they test.
bitLength := u.Bits() // scans all of u.u, past u.l
packed := append([]byte(nil), u.Bytes()...) // u.u[:u.l] only
packed = packed[:(bitLength+7)/8] // can exceed cap(packed) -> panic
Nits
Checked and fine
|
1b55705 to
0b0a9d8
Compare
There was a problem hiding this comment.
Pull request overview
This PR updates Caplin/CL code to match consensus-specs v1.7.0-alpha.12 for the Gloas fork, adding progressive SSZ decoding/hashing paths, tightening resource bounds/validation for progressive layouts, and refreshing/expanding consensus spec tests and fixtures accordingly.
Changes:
- Introduce progressive SSZ hashing/merkleization and progressive list/container helpers across Gloas-related CL types and beacon-state hashing.
- Add config-aware bounds and malformed-input rejection (nested nil checks, size/cap validations) across JSON/SSZ decode paths and network/pool handlers.
- Update and extend consensus spectests (fixtures, gossip, rewards, static SSZ helpers, light-client proofs) to alpha.12.
Reviewed changes
Copilot reviewed 65 out of 65 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test-fixtures.json | Bumps consensus-spec fixtures from alpha.11 to alpha.12. |
| cl/validator/devvalidator/aggregate.go | Sets fork version on aggregate-and-proof messages/signatures. |
| cl/validator/devvalidator/aggregate_test.go | Adds coverage ensuring aggregate attestation hashing matches slot fork version. |
| cl/transition/machine/block.go | Adds Gloas operation-count validation before processing operations. |
| cl/transition/machine/block_gloas_test.go | Tests oversized Gloas operation list rejection. |
| cl/transition/impl/eth2/operations.go | Bounds execution-request counts in ApplyParentExecutionPayload. |
| cl/transition/impl/eth2/operations_gloas_test.go | Updates builder deposit request test vectors for new required fields. |
| cl/ssz/decode.go | Fixes static-sized SSZ element decode to slice exact size. |
| cl/spectest/consensus_tests/ssz_static_helpers.go | Adds Go SSZ implementations for formerly-unimplemented static-helper types. |
| cl/spectest/consensus_tests/rewards.go | Implements rewards spectest runner (removes prior skip) and delta decoding/compare. |
| cl/spectest/consensus_tests/rewards_test.go | Adds unit tests for reward-deltas SSZ decoding bounds/valid cases. |
| cl/spectest/consensus_tests/light_client.go | Adds handler support for Gloas execution block hash merkle proof. |
| cl/spectest/consensus_tests/gossip.go | Implements additional gossip spectest validators (BLS-to-exec, sync committee, contributions). |
| cl/spectest/consensus_tests/gossip_bounds_test.go | Adds bound/overflow tests for new gossip helpers. |
| cl/spectest/consensus_tests/appendix.go | Wires new/expanded formats and enables new static-helper handlers. |
| cl/phase1/network/services/aggregate_and_proof_service.go | Adds nested nil checks; enforces attestation config validation and version propagation. |
| cl/phase1/network/services/aggregate_and_proof_service_test.go | Adds coverage for malformed nested aggregate-and-proof inputs. |
| cl/phase1/forkchoice/on_attester_slashing.go | Uses centralized indexed-attestation indices validation. |
| cl/phase1/forkchoice/checkpoint_state.go | Uses config/version-aware indices validation for indexed attestations. |
| cl/phase1/core/state/raw/setters.go | Invalidates roots and toggles validator hashing mode when crossing Gloas boundary. |
| cl/phase1/core/state/raw/hashing.go | Implements Gloas progressive container roots/proofs; switches some leaves to progressive hashing. |
| cl/phase1/core/state/raw/hashing_gloas_test.go | Adds tests for progressive hashing and SetVersion invalidation semantics. |
| cl/phase1/core/state/epbs.go | Updates builder credential logic and builder deposit request processing rules. |
| cl/phase1/core/state/epbs_test.go | Updates tests for configured builder prefix and updated withdrawable-epoch behavior. |
| cl/phase1/core/state/accessors.go | Adds config/version-aware ValidateIndexedAttestationIndices helper and uses it. |
| cl/phase1/core/state/accessors_gloas_test.go | Tests oversized Gloas attesting indices are rejected before lookup. |
| cl/merkle_tree/merkle_root.go | Adds progressive container roots/proofs and progressive list roots; adds local hashPair. |
| cl/merkle_tree/merkle_root_test.go | Adds coverage for oversized progressive-container proof schema rejection. |
| cl/cltypes/solid/vector_test.go | Adjusts test helper to ensure aggregation bitlist has delimiter bit set. |
| cl/cltypes/solid/validator_set.go | Adds progressive validator-set hashing with segmented progressive merkle caches. |
| cl/cltypes/solid/validator_set_progressive_test.go | Adds progressive root reference tests and mode-switching tests. |
| cl/cltypes/solid/uint64_raw_list.go | Adds SSZ decode bound checks and progressive hash. |
| cl/cltypes/solid/uint64_raw_list_test.go | Tests uint64 raw list decode rejects partial elements/over-limit. |
| cl/cltypes/solid/transactions.go | Adds progressive hashing for transactions list. |
| cl/cltypes/solid/participation_bitlist.go | Adds progressive hashing for participation bitlist. |
| cl/cltypes/solid/list_ssz.go | Introduces progressive list mode with resource-guard decode limits and progressive hashing. |
| cl/cltypes/solid/list_ssz_test.go | Adds tests ensuring progressive list decode enforces the configured limit. |
| cl/cltypes/solid/byte_list.go | Adds progressive hashing for ByteListSSZ. |
| cl/cltypes/solid/bitvector.go | Adds strict size/unused-bits validation and a ValidateSize helper. |
| cl/cltypes/solid/bitvector_test.go | Adds tests for bitvector invalid size/unused bits rejection. |
| cl/cltypes/solid/bitlist.go | Adds progressive hashing and enforces canonical/limited SSZ decoding. |
| cl/cltypes/solid/bitlist_test.go | Adds tests for non-canonical bitlist encoding and limit acceptance. |
| cl/cltypes/solid/attestation.go | Adds version tracking, config validation, and progressive hashing for attestations. |
| cl/cltypes/solid/attestation_config_test.go | Adds tests for config-aware committee-bits size checks and JSON normalization. |
| cl/cltypes/slashings.go | Adds progressive hashing for AttesterSlashing. |
| cl/cltypes/partial_data_column.go | Switches some Gloas sidecar lists to progressive list types and progressive hashing. |
| cl/cltypes/light_client.go | Updates/extends Gloas light-client proof branch sizes. |
| cl/cltypes/indexed_attestation.go | Adds version tracking and progressive hashing for indexed attestations. |
| cl/cltypes/gloas_progressive_hash_test.go | Adds a regression test for progressive block/body roots with a fixed encoded fixture. |
| cl/cltypes/execution_requests.go | Makes execution request lists progressive-aware and adds JSON nil-element validation/coalescing. |
| cl/cltypes/eth1_block.go | Implements Gloas-specific progressive hashing for execution payload header. |
| cl/cltypes/epbs_payload.go | Switches several hashes/proofs to progressive containers; adds JSON validation for payload bids. |
| cl/cltypes/epbs_payload_test.go | Adds tests for preserving progressive limits through decode/clone/JSON paths. |
| cl/cltypes/column_sidecar.go | Uses progressive list types for Gloas columns/proofs and resets on DecodeSSZ. |
| cl/cltypes/beacon_block.go | Refactors per-config limits, adds Gloas progressive list initialization, and implements Gloas progressive body hashing/proofs. |
| cl/cltypes/beacon_block_test.go | Adds tests for null-required-field rejection and config-aware progressive list limits. |
| cl/cltypes/beacon_block_blinded.go | Applies attestation config validation post-decode. |
| cl/cltypes/aggregate.go | Adds version propagation into nested objects and JSON handling that preserves versions. |
| cl/cltypes/aggregate_gloas_test.go | Adds tests around version preservation and nil-nesting behavior for aggregates. |
| cl/clparams/devgenesis/devgenesis.go | Initializes Gloas genesis with progressive validators root and execution payload bid/requests root. |
| cl/clparams/devgenesis/devgenesis_test.go | Adds Gloas genesis tests validating progressive validators root and bid/body root wiring. |
| cl/clparams/config.go | Updates mainnet Gloas builder limits/prefix/min delay to alpha.12 values. |
| cl/beacon/handler/pool.go | Adds aggregate-and-proof request validation and version/config-based checks before gossip. |
| cl/beacon/handler/pool_test.go | Updates attester slashing test setup to use constructor with version-aware fields. |
| cl/beacon/handler/block_production.go | Validates attestations for config/version when publishing blinded blocks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 65 out of 65 changed files in this pull request and generated no new comments.
Suppressed comments (2)
cl/phase1/core/state/raw/hashing.go:179
- sync.WaitGroup has no Go method; this block won’t compile. Use wg.Add(1) + a goroutine with defer wg.Done() (and pass idx/job into the closure).
cl/merkle_tree/merkle_root.go:250 - progressiveSchemaRoots allocates and calls BytesRoot even for the common case of 32-byte chunks (e.g., state leaves). For len==32, the SSZ root is the chunk itself, so this can be a straight copy to avoid per-field allocations/work in ProgressiveContainerRoot/Proof paths.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 65 out of 65 changed files in this pull request and generated no new comments.
Suppressed comments (1)
cl/phase1/core/state/accessors.go:215
- ValidateIndexedAttestationIndices multiplies two uint64 config limits (MaxValidatorsPerCommittee * MaxCommitteesPerSlot) without overflow protection. With large custom configs this can wrap and produce an incorrect (smaller) limit, causing mis-validation of attesting indices sizes. Use a saturating multiply (cap at MaxUint64) before comparing against inds.Length().
a37a997 to
5b45171
Compare
|
Thanks for the thorough review. I addressed the correctness and robustness items you called out:
The gossip spectest-handler consolidation is intentionally left out of this PR because it is a broader production-service refactor; this PR keeps the fixture coverage without mixing that architectural change into the alpha.12 catch-up. The branch is now rebased onto current main. Focused CL/spectest/network tests, repeated lint, and the required Erigon/integration builds pass locally. The refreshed GitHub checks are running now. Please take another look when convenient. |
yperbasis
left a comment
There was a problem hiding this comment.
Approving. Verified locally beyond reading the diff: full CL spectest suite against the new alpha.12 fixtures passes (8923 cases, 0 failures; all 1258 Gloas cases run with 0 skips), unit tests green in every touched package, and I hand-checked the operation-count asserts, builder-deposit semantics and the three light-client branch depths against the v1.7.0-alpha.12 spec text.
Findings below. Only the first is worth fixing before merge; nothing touches consensus correctness.
Null blob commitment panics the standalone bid handler
ExecutionPayloadBid.UnmarshalJSON rejects a missing or null blob_kzg_commitments list but accepts [null], producing a one-element list holding a nil *KZGCommitment. HashSSZ then nil-derefs — there is no recover() anywhere in cl/merkle_tree.
Reachable path from POST /eth/v1/beacon/execution_payload_bid:
ProcessMessage → validateBidStateless (passes: only checks Len() against the blob schedule) → validateHighestBid → matchingProposerPreferences returns not-available → queuePendingBid → pendingBidKeyFor (execution_payload_bid_service.go:523, root, _ := msg.HashSSZ()) → panic.
Scope: REST only. SSZ static-list decode allocates every element via Clone(), so gossip cannot produce nil members. It is also not a node crash — this runs on the net/http handler goroutine, which recovers per connection, and the panic happens during key computation before queue insertion, so a poisoned bid never reaches the background bid loop. Net effect is a dropped connection plus a stack trace in the logs.
Re-encode is already defended, incidentally: ssz2.MarshalSSZ recovers at cl/ssz/encode.go:73 and returns an ordinary error, so the gossip-publish branch cleanly 500s. Only the hash path is exposed.
Fix: validate every commitment in ExecutionPayloadBid.UnmarshalJSON and return 400, matching what the body decoder in beacon_block.go already does for this exact shape.
Arbitrary decode cap on a spec-unbounded deposits list
progressiveDecodeLimit turns the Electra max of 8192 into a hard decode cap of 16384. Gloas defines requests.deposits as an unbounded ProgressiveList, and it is the one request list apply_parent_execution_payload deliberately does not assert on — the other four have real transition asserts backstopping their 2x caps, which is why the arbitrary bound is only visible here.
Reproduced: 16384 deposits decode; 16385 fails with ErrTooBigList at 3,145,940 encoded bytes, well under MaxChunkSize (15 MiB).
Practically unreachable on mainnet — roughly 16k deposit-contract calls at about 50k gas each is on the order of 800M gas against a ~45M limit — so this is hygiene rather than a live bug. Worth noting that test_deposit_requests_greater_than_electra_max uses 8193 deposits, which is under the cap and passes, consistent with the clean suite run above; the test establishes that the field is unbounded, not that we reject a conformance case.
Fix as suggested: derive the bound from the enclosing message-size limit rather than semanticLimit * 2. For static progressive lists the decoder already knows len(buf), so len(buf)/bytesPerElement is the natural bound and the message-size limit does the real work.
Smaller items
BeaconBody.DecodeSSZ(cl/cltypes/beacon_block.go:963) resets lists to progressive when decoding a Gloas body, but decoding a pre-Gloas body into a struct that previously held a Gloas body keeps the progressive lists and would hash with the wrong scheme.ExecutionRequests.DecodeSSZhandles both directions. Latent today — every call site constructs bodies fresh viaNewBeaconBody/NewSignedBeaconBlock— but cheap to make symmetric.- Dead check in
aggregate_and_proof_service.go: the explicitaggregate.CommitteeBits == niltest afterValidateForConfigis unreachable, sinceValidateForConfigalready rejects nil committee bits at Electra and later. hashPairuses stdlibcrypto/sha256while the rest ofcl/merkle_treeuses the pooledcommon/crypto.Sha256. It sits on the state-root and body-root hot paths.maxPayloadAttestationsForConfigreads the globalclparams.GetBeaconConfig()as a fallback layer; the explicitbeaconCfgparameter plus the mainnet constant would be more deterministic in tests. Related: theptcSizefallback block is copy-pasted three times inbeacon_block.go, andExecutionRequests.UnmarshalJSONrepeats five near-identical null-element blocks — both want a small helper.- PR description is stale: it says head is
b3fdfc34de, but3cc88f219cand5b45171797landed after (both reviewed, both fine). Branch is also 10 commits behind main; CI is green on the current base.
Summary
Catch Caplin up to the Gloas alpha.12 consensus fixtures and progressive SSZ layouts used by glamsterdam devnet-7.
This PR contains the progressive SSZ and consensus-spec catch-up only. The related runtime/external-EL fixes are independently based on main in #22683.
Changes
Validation
go test ./cl/spectest ./cl/cltypes ./cl/cltypes/solid ./cl/merkle_tree ./cl/clparams/devgenesis ./cl/beacon/handler -count=1go test -race ./cl/cltypes ./cl/cltypes/solid -count=1make lintrepeated clean: 0 issuesmake erigon integrationgit diff --checkThe commits were replayed onto main at
5d00a6a895. Current head isb3fdfc34de.Adversarial review
Two independent subagent reviews converged after repeated fix/review rounds. Findings fixed during the final rounds included:
uint64configuration limits wrapping during conversion tointThe final audits also covered max+1 decode-before-transition behavior, custom/future configs, clone/copy/cache behavior, malformed cross-boundary objects, and pre-Gloas reverse paths.
GitHub Copilot reviewed all 65 changed files. Its actionable request-index and typo findings were fixed, while the
sync.WaitGroup.Gocomment was rejected because this repository targets Go 1.25.7 where that API is available. Copilot re-reviewedb3fdfc34deand generated no new comments.